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Digital Fingerprints and Watermarks for Images 



This invention relates to the application of digital fingerprints and/or 
watermarks to images. 

5 Video fingerprinting can be used for video content identification and 

associated applications. It is desirable tbat the video fingerprint recognition can proceed 
successfiiUy despite certain distortions to the video content, such as compression, scaling, or 
translation. 

Bacl^round 

10 A typical video fingerprint extraction scheme consists of a first stage of 

extracting features firom video frames. Many different features have been proposed for use in 
generating fingerprints including: DC (mean Ixmainance) values, luminance and/or 
chrominance histograms, and edges. A common aspect of these schemes is that images are 
usxially divided up into regions (e.g. rectaagular blocks) and features extracted for each 

15 region. Such a fingerprinting scheme is disclosed in International Patent AppUcation 
WO-A-02/065782. 

The division of an image frame into regions causes recognition problems if the 
images undergo scaUng or translational shifts. The fixed grid used to divide up the image 
results in image regions containing different content for a scaled/translated image than for the 
20 unscaled/unshifted original. This causes differences in the resulting fingerprints that will 
often prevent the scaled/translated content from being recognised. 

It is an object of the present invention to address the above mentioned 

25 disadvantages. 

According to a first aspect of the present invention a method of 
applying/detecting a digital fingerprint to/in an image comprises: 

dividing the image into regions, the location and size of the regions being 
based on a content of the image; and 
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applying/detecting the digital fingerprint 

According to a second aspect of the present invention a method of dividing an 
image into regions for the application/detection of a digital fingerprint comprises selecting 
the location and size of the regions according to a conteat of the image. 

Preferably, the image is divided based on the location of a centroid of the 
image. Preferably, a grid for dividing the image is ceatred substantially at a location of the 
centroid of the image. 

Preferably, horizontal and vertical sizes of divisions of the grid are 
independent of one another. 

Preferably, the size of horizontal and/or vertical divisions is based on a content 
of portions of the image, preferably based on centroids of portions of the image. 

The method advantageously allows a grid location and size to be calculated 
based on image content, thereby allowmg scaling and translation of the image without 
affecting the watermarking thereof. 

The regions may overlap, either horizontally and/or vertically. It has been 
found that overlapping regions can be used with advantageously better results. 

According to a third aspect of the invention a method of embedding/detecting 
a digital watermark into an image comprises selecting a size and/or location of a watermark 
pattern based on a content of the image, and applying/detecting the watermark. 

Preferably, a location of the watermark, which is preferably present as a grid 
element, is chosen according to a location of a centroid of the image. Preferably, a size of the 
watermark, preferably a size of grid for the watermark present as grid elements or in grid 
elements, is chosen according to a content of portions of the image, preferably centroids of 
portions of the image. 

According to a foixrth aspect of the invention there is provided a computing 
device programmed to operate the method of the first and/or second aspect 

According to a fifth aspect of the invention a computer program product is 
operable to perform the method of the first and^or the second aspect of the present invention. 

All of the features described herein may be combined with any of the above 
aspects, in any combination. 
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For a better understanding of the invention and to show how the same may be 
brought into effect, specific embodiments wiU now be described, by way of example, with 
reference to the acconqsanying drawings, in wdiich: 

Figure 1 is a sdiematic diagram of an image frame showing a centroid; 

Figure 2 is an unage haying a grid of blocks appUed thereto; 

Figure 3 is the image of Figure 2,after horizontal scaling and vertical shifting; 

Figure 4 shows bit error rates (BER) for fingerprints of a first example video 
clip and the same vide cUp after scaling/shifting; 

Figure 5 shows BER for a second example video chp after scaUng/shifling; 

Figure 6 shows a saiiq)le image with a "subtitle" applied; 

Figure 7 shows BER for the first example image with a subtitle added; and 

Figure 8 shows BER for Ihe second example image with a subtitle added 



invention is a technique of deriving fingerprints for image/video content 
that improves the recognition rate in cases where the content is scaled or translated. It is also 
described how the same idea can be apphed to watermarking in order to help achieve 
detection in scaled/shifted versions of a watermarked image. 

The key idea in this invention disclosure is to divide images into regions in a 
20 manner dependent upon the image content itself. In other words, the grid used to divide 

frames into regions should be determined from the image content. In this way, Ihe grid can be 
made to scale and shift identically to the image content, resulting in fingerprints for 
scaled/shifted images that are as close as possible to those of the original images. 

A content dependent division of the image can be achieved using centroid 
25 (centre of mass) calculations. The centroid of an image region with luminance i(jc, y) is the 
point (x,y) where: 

Note that these can be computed as: 

X y X y 
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i.e. the calculation is esseatiaUy one dimmsional and the number of multiplications can be 
minimised by (for x)suniming I(x,y) first over the y direction. The centroid is thus a 
relatively cheap computation. 

The centroid has the property of shifting identicaUy to the image content: 

* y 



and also of scaling identically to Ihe image content 

X y 



' TLiOc',y) 

y y 



These calculations assume that the summations are taken over the entire image 
content in up-scaled and shifted versions of it. In practice up-scaling and shifting will be 
combined with cropping in order to preserve the original image dimensions. Cropping 
involves removing sections of the image from one or more edges to achieve a required ratio 
of side lengths for the image. This cropping results in errors in the centroid location, which 
prevents perfect invaiiance to up-scaling and shifting. 

Similarly, down-scaling will in practice be combined wilh padding with new 
content (by adding additional data to achieve a required ratio of side lengths) in order to 
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retain the original image dimensions. Scale invariance is still achieved if the padding is made 
with pixels of value zero, such that they do not contribute in the summations above. 

Despite these imperfections in a practical scheme, it will be seen that tiie 
centroid shifts and scales sufficiently accurately to achieve considerable robustness in the 
5 resulting fingerprints. 

Using centroids to provide a scale/shift robust division of the image will be 
explained for the exan:^>le of spUtting an image into a grid consisting of a nimiber of 
rectangular blocks. This is d^ired, for example, by the fingerprint scheme in 
WO-A-02/065782 Ihe contents of which are incorporated herein by reference, where the DC 
10 differences between these blocks are then used to derive Ihe fingerprint bits. The process 
consists of the following steps: 

Calculate the cratroid C (x, y) of the whole image - The grid will be centred 
upon this point, providing robustness to translational shifts of the image 
Calculate the centroid in the horizontal direction xuft of all the image pixels left of C. See 
15 Figure 1. 

Calculate also the centroid Xnght of all image pixels right of C, the centroid 
yabove in the vertical direction of all the image pixels above C, and the centroid yteiow of all the 
image pixels below C. 

Define block boundaries at: 

20 Horizontal direction: x-^^, and 5c + ^^^'^ for /e {0,1,2,... ,iNr} 

N N 

Vertical direction: J- ^^'^^ and J + ^^i^^ for /e {0,1,2,...,M} 

M M 

where a is a multiple chosen to control the total size of the resulting grid of blocks. For 
example, setting a=2 results in a grid spanning most of the original image, since the centroids 
of typical image content are usually fairly central. The resulting grid consists of 2A4x2iV 
25 blocks. 

In figure 1 xuft is the centroid in the horizontal direction of the portion of the 
image left of the total image centroid C (x, y) 

Figure 2 shows an example grid of blocks formed using the above procedure 
for a =2, and M=N^3, Figure 3 shows the grid formed for the same image scaled horizontally 
30 by a factor of 1 . 1, and shifted vertically by 20 pixels. It can be seen that the image content of 
each block is similar in both cases, and much more so than if a fixed grid had been used. 
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The above procedure was used to form blocks from vMch fingerprints were 
subsequently derived as in WO-A-02/065782. Figures 4 and 5 show the fingerprint bit error 
rates (BER) between fingerprints of the original video content, and fingerprints from 
scaled/shifted versions of the same content. For comparison, the fingerprint bit error rates for 
5 fixed blocks of 144x80 pixels are also shown; the improvemeat of the new centroid scheme 
is clear. 

Note that tiie BERs due to shifts and up-scales are lower for the 'interv' video 
cUp in figure 4 than they are for the *kitchgrass' cUp in figure 5. This is because Ihe former 
has a sUghfly wider black border at its perimeter. The near-zero intensity values of such a 
10 border do not mfluence Ihe centroid position, thus the BER only begins to rise at larger shifts 
or up-scales where the effect of cropping begins to take effect Note also that the BERs for 
down-scaling are lower than for iip-scaling, as e>q)ected, as no cropping takes place under 
down-scaling. 

An additional advantage of the presented scheme is that it is robust to different 
15 scale factors in the horizontal and vertical directions, i.e. it is robust to aspect ratio chaages. 
Application to Watermarking 

Scaling and shifting of an image can also prevent detection of a watermark 
contained in that image. Suppose, as in many watermarking schemes (see WO-A-99/45705, 
for example), that Ihe watermark pattern can be formulated as a tile of noise samples in the 

20 spatial domain. This watermark pattern is added to the original image in order to create the 
watermarked image. The invention presented above allows the size and shift of the embedded 
watermark pattern to be set by properties of the image content itself (e.g. centroids). The 
watennark detector can automatically find the appropriate scale and shift of the watermark 
pattern in the received image by repeating the procedure. 

25 In practice, watermark detection is typically very sensitive to scaling, and the 

accuracy of the scaling indicated by (e.g.) the centroids may not be great enough to allow 
detection of the watermark. The invention does, however, provide a first estimate of the scale 
factor, and this can be used to reduce the number of scale factors that need to be searched in 
order to detect the watermark. Thus the invention can still provide an advantage in terms of 

30 reduced computation in the watermark detector. 

The metiiod can be appUed equally well to the application or detection of a 
fingerprint or watermark. 

Further Experiments 
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This section describes finther e>q)erimeats concerning Hie above centroid 
based scheme for improved scale and shift robustness. 
Subtitles 

The addition of a subtifle to a video clip adds areas of bright white pixels to 
5 the bottom of the images. E3q)eriments were perfonned to assess whether Ihis disturbs Ihe 
centroid positions sufficiently to cause a significant BER increase \^en matching 
fingerprints. 

Subtifles were ^simulated' by adding high density *salt and pepper' noise in 
tiie area where subtitles woiild be added. An example is shown at Figure 6: 
10 This was found to cause a BER. of around 7% for the centroid scheme 

compared to around 4% for the original scheme of fixed blocks. The position and size of Ihe 
grid was found to be only slightly aflfected by the addition of the 'subtilie'. The miriimfll 
impact of the subtitle is simply due to it being only a small firaction of the total number of 
image pixels; in these experiments the subtitle occupied 5% of the image height, and 60% of 
15 Ihe image width, i.e. only 3% of pixels. 

Overlapping blocks 

The robustness to shifts and scaling is highly dependent upon the size of the 
blocks psed in calculating the DC differences. This is true for both the fixed block and 
centroid schemes. In order to increase the effective size of blocks an overlap was introduced 
20 as follows: 

Each block is extended horizontally such that it covers 50% of each block 
either side, and likewise vertically. For blocks at the perimeter of the image, the extension is 
made just to the image edge 

A banning window is appUed to the pixel values of the extended block 
25 The DC is calculated 

The danger of overiapping the blocks is that each pixel contributes to multiple 
fingerprint bits, and this may increase the correlation between them. If the fingerprint bits 
from a given frame are correlated then they contain redundancy, i.e. we effectively have 
fewer fingerprint bits, which will lead to higher false positive rates (matching fingeiprinis 
30 from different content). The above overlap method was found to generate fingerprints which 
still appeared very similar to those of the non-overlapping blocks. From this it seems 
reasonable to assume that the resulting fingerprints will still be sufiELciendy discriminating 
between different content. 
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The fingerprint matching results for the xisual set of attacks are shown at 
Figures 7 and 8 for two example video clips (dashed line wilh stars: fixed non-overlapping 
blocks, solid line wilh triangles: centroid scheme with non-overlapping blocks, solid hne wilh 
stars: centroid scheme with overlapping blocks). It can be seen that the overlap of blocks 
5 results in similar or sUghtly improved BER in all cases. 



